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(57) ABSTRACT 

The present invention relates to an interactive digital system 
enabling viewers full and active participation in experienc- 
ing a live broadcast event. Particularly, the presentation of 
the live event is personalized for the viewer through the 
provision of various options, including multiple video 
streams, associated with different camera angles, for 
example, and integrated audio and graphics segments. Fur- 
ther, information obtained from related Web sites can be 
integrated into the live program. Various video and audio 
streams are collected from a live event and forwarded to a 
central control studio. Graphics are created at the central 
studio on a personal computer or chyron device. After 
receiving the video, audio and graphics signals, the signals 
are digitized and compressed in digital compressors. These 
signals are then combined with special data codes into a 
"digital package," and subsequently, transmitted over a 
cable distribution system. Once received at a viewer home, 
the signals are received and processed in an interactive 
digital cable box. Selections of the video, audio, graphics 
displays and/or Web pages can be made as a function of 
immediate viewer entries, or to interrogatory responses 
presented at the beginning or during the program, or based 
on a prestored viewer profile. Once a decision is made to 
switch from one video to another video option, the digital 
switch is performed seamlessly. The digital interactive sys- 
tem is based upon seamless branches which occur in the 
course of full-motion video. 
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DIGITAL INTERACTIVE SYSTEM FOR 
PROVIDING FULL INTERACTIVITY WITH LIVE 
PROGRAMMING EVENTS 

CROSS REFERENCE TO RELATED 
APPLICATIONS 

[0001] This application is a continuation of U.S. applica- 
tion Ser. No.08/815,168 filed Mar. 11, 1997, which is a 
continuation-in-part of application Ser. No. 08/598,382 filed 
Feb. 8, 1996 now U.S. Pat. No. 5,861,881, which is a 
continuation-in-part of application Ser. No. 08/443,607 filed 
May 18, 1995, now U.S. Pat. No. 5,724,091, which is 
continuation-in-part of application Ser. No. 08/166,608 filed 
Dec. 13, 1993, now abandoned, which in turn is a continu- 
ation of application Ser. No. 07/797,298 filed Nov. 25, 1991, 
now abandoned, each of which are hereby incorporated 
herein by reference in their entirety. 

BACKGROUND OF THE INVENTION 

[0002] Interactive video and audio presentation systems 
are currently being introduced into the entertainment and 
educational industries. A prominent interactive technology 
that has been applied successfully in these industries is 
based on providing interactivity in a one-way system 
through the provision of multiple time-synched parallel 
channels of information. For example, commonly owned 
Freeman et al. patents, U.S. Pat. Nos. 4,264,925 and 4,264, 
924, which provide both audio and video interactivity, 
disclose interactive television systems where switching 
among multiple broadcast or cable channels based on viewer 
selections provides an interactive capability. 

[0003] These systems have been enhanced to include 
memory functions using computer logic and memory, where 
selection of system responses played to the viewer are based 
on the processing and storage of subscriber responses, as 
disclosed in Freeman patent, U.S. Pat. No. 4,507,680. 

[0004] The benefits of providing interactivity through the 
use of different audio responses is disclosed in Freeman, 
U.S. Pat. Nos. 4,847,698, 4,847,699 and 4,847,700. These 
television systems provide a common video signal accom- 
panied by several synchronized audio channels to provide 
content related user selectable responses. The audio signals 
produce different audio responses, and in some cases, these 
are syllable synched to a first audio script and to the video 
signal (such as to a person or character on a display), 
providing the perception that the person's or character's 
mouth movements match the spoken words. 

[0005] Interactivity is brought to the classroom in the 
Freeman U.S. Pat. No. 5,537,141. The distance learning 
system claimed in this application enhances the classroom 
educational experience through an innovative use of inter- 
active technology over transmission independent media. 
When an instructor, either broadcast live on video or dis- 
played from videotape, asks a question, each and every 
student responds, preferably by entering a response on a 
remote handset, and each student immediately receives a 
distinct and substantive audio response to his or her unique 
selection. The individualization of audio response from the 
interactive program is a major aspect of the invention. 

[0006] Individualization of audio is brought to the home 
based on the technology disclosed in Freeman U.S. Pat. No. 



5,585,858. This system provides a program that can be 
watched on any conventional television set or multimedia 
computer as a normal program. But if the viewer has a 
special interactive program box connected to the television, 
he or she can experience a fully functional interactive 
program. Each interactive viewer enjoys personalized audio 
responses and video graphics overlayed on the screen. The 
interactive program can be provided to television sets or to 
computers by cable, direct broadcast satellite, television 
broadcast or other transmission means, and can be analog or 
digital. 

[0007] However, what is needed is an interactive presen- 
tation system for providing true video, audio and graphics 
interactivity with digital programs broadcast live. Such a 
system must efficiently package all the digital elements of 
the live interactive program at a centralized control studio 
and allow viewers at home to receive personalized interac- 
tive programming. 

SUMMARY OF THE INVENTION 

[0008] In accordance with the invention, there is provided 
an interactive digital system allowing the viewer active 
participation in selecting digital video streams, associated 
with different camera angles, for example, and integrated 
audio and/or graphics segments. Further, Web pages from 
Internet Web sites can be integrated into the program. The 
invention is particularly suited for the environment of live 
events, such as the broadcast of live sporting events. The 
viewer can appear to direct the camera shots by instantly 
changing among various camera angles, choose player inter- 
views, or display associated statistical data on the team or 
players via graphics. In this manner, the system allows the 
individual subscriber to act as if he or she has control over 
how the program is directed and presented on their personal 
television set. In addition to selecting different camera 
angles, various audio options, closeups, slow motion, 
replays, graphics overlays, graphics or audio from Web sites, 
etc., are all possible. Further, games can be integrated with 
the live sports programming to increase viewer interest. 

[0009] Thus, viewers can customize the content of pro- 
grams. The interactive digital programming of the present 
invention is particularly advantageous for viewing live 
sporting events. Viewers are not limited to selecting from 
multiple camera angles, but may also call up player statistics 
on demand, listen to selected player interviews, etc. Cam- 
eras can be focused on different segments of an event. 
Further, video options could include video replay, slow 
motion effects, isolation on a particular player or group, etc. 
Changes are seamless, thereby adding to the effect that the 
viewer is directing the television show just as a director now 
does from a control room. 

[0010] This "director" role by the viewer is possible due to 
the interactive technology of the present invention and also 
due to the digital compression and transmission scheme 
which allows for much greater information throughput over 
a given bandwidth, allowing viewers to choose from angles 
that are already available but presently cut by the director. 

[0011] The digital interactive system is based upon 
branches which occur in the course of the full-motion video. 
Branches are real-time parallel paths that may be other 
full-motion video segments, graphics which are integrated 
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into the video, audio segments, and/or retrieved Web pages 
which are integrated into the live event. 

[0012] Sometimes, the interactive digital system will act 
upon the viewer's response immediately; other times, it will 
utilize ACTV's unique "profiling" concept to act upon the 
response later. This technology enables the system to 
"remember" the viewer's responses and desires, and inte- 
grate them into the video, audio, graphics and/or Web site 
information at a later point. For example, the viewer could 
specify at the beginning of a football game to isolate the 
offensive quarterback of a particular team. Thus, whenever 
the team of choice is on offense, the video isolation of the 
quarterback is displayed to the viewer automatically. Or, 
based on how a viewer has selected camera angles, replays, 
etc., over the past five minutes, the system acts to mimic 
these selections at later times during the program. The 
system of the present invention "learns" from the viewer 
how they want to view the game, and thus, continues viewer 
selection sequences made earlier. 

[0013] At the source, the present invention comprises a 
plurality of video cameras, each of the video cameras 
relaying a different predetermined view of an event. The 
video signals corresponding to the different cameras are 
forwarded to a central control studio. Further, one or more 
audio signals or graphic statistical overlays can be collected 
and sent to a central control studio. After receiving the video, 
audio, and graphics signals at the central control studio, 
these signals are digitized and compressed in digital video 
and audio compressors. These signals are then combined 
with special data codes into a "digital package," and sub- 
sequently, transmitted over a cable distribution system. The 
special data codes are the keys to unlocking the interactive 
potential of the program. 

[0014] The digital program signals are transmitted to a 
receive site by any suitable transmission means. Once 
received by a receive antenna, the digital program signals 
are passed along on a digital cable television distribution 
system to the viewer homes. Further, some other signals or 
commercials can be inserted at the local head end. The 
signals are received and processed in a digital cable box. 
Selections of the video, audio, graphics display and/or Web 
pages can be made as a function of immediate viewer 
entries, or to interrogatory responses presented at the begin- 
ning or during the program, or based on a prestored viewer 
profile. Once a decision is made to switch from one video 
option to another video option, the digital switch is per- 
formed seamlessly. 

[0015] As mentioned above, the program at predetermined 
times or immediately upon user entry can retrieve and 
branch to informative segments from Web sites. For 
example, a viewer watching a sporting event, through the 
system of the present invention, can receive a stream of Web 
pages which provide additional, specific information relat- 
ing to a favorite player, team or perhaps the remaining 
schedule for the sports team, as examples. In addition, users 
can take advantage of the two-way capabilities of the 
Internet to respond to polls or to link to additional sites. 

[0016] Another Internet-based application allows adver- 
tisers to speak more directly to consumers by directly 
sending Web pages to the consumer instead of merely 
displaying Web addresses in their commercials. The particu- 
lar advertising information from Web sites can be targeted to 



viewers based on the viewer profile, stored either in the 
digital set top box or at the cable headend. Alternatively, 
Web site access can be initiated by the viewer by simply 
clicking on the remote during the commercial. Thus, viewers 
have the capability to individually select Web sites if they 
want more information from advertisers, for example. 

[0017] The video programming and corresponding Inter- 
net pages can be viewed either on personal computers 
equipped with a television card on special digital cable 
boxes with stored interactive Internet application software 
providing Internet access, or on digital television sets, all of 
which would utilize the specialized TV/Internet software of 
the present invention. 

[0018] The present invention also has applications for 
other types of programming. For example, viewers can 
direct the scenes of a murder mystery. Switching from one 
scene to another can be done seamlessly without noticeable 
effect on the viewer. Further, the present invention can be 
used for any kind of live or pre-recorded event. For instance, 
a music concert or a political convention can be enhanced in 
the manner of the invention. 

[0019] Accordingly, a primary objective of this invention 
is providing an enhanced digital live program allowing the 
display to be tailored to the user's desires, choices or 
interests. 

[0020] It is an object of this invention to personalize and 
enhance live sporting events for the viewer. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0021] FIG. 1 is a diagram of the network and equipment 
for providing live digital programming. 

[0022] FIG. 2 is a block diagram of an interactive digital 
cable box allowing seamless switching between video sig- 
nals. 

[0023] FIG. 3 is a block diagram of an alternative dual- 
tuner interactive digital cable box allowing seamless switch- 
ing between video signals. 

[0024] FIG. 4 is a block diagram of another alternative 
interactive digital cable box allowing seamless switching 
between video signals. 

[0025] FIG. 5 is a time diagram showing a representation 
of trigger points and corresponding video, audio and/or 
graphics segments, one or a combination of which are 
selected for presentation to the subscriber immediately after 
the execution of the trigger point function. 

[0026] FIG. 6 is a block diagram of an alternative embodi- 
ment of the interactive system including Internet access. 

[0027] FIG. 7 is a block diagram of the two-way configu- 
ration of the system. 

DETAILED DESCRIPTION OF THE 
INVENTION 

[0028] The present invention is an interactive digital sys- 
tem 1 for producing a powerful personalized program allow- 
ing the home viewer an expanded set of programming 
options. Digital TV streams are put into digital packages 
made up of video, audio, data codes and graphics, and are 
used to provide personalized responses to viewer selections. 
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Such responses can be further enhanced by allowing access 
to Internet Web sites 170. In this manner, sports such as golf, 
football, baseball, basketball, etc. can now be watched with 
greater interest and involvement. However, even further 
enhanced interactivity is possible with the present invention 
due to the provision of various profiling and memory 
features. 

[0029] As shown in FIG. 1, the present invention begins 
with the gathering of several possible video streams by way 
of cameras 100, strategically located at a sporting event 10, 
for example. Currently, many cameras 100 are employed at 
a sporting event 10. Super Bowl coverage, for example, 
typically encompasses 25 to 30 cameras. These live video 
streams can be integrated with recorded video streams 
which, for example, could include highlights from the 
current game or past games, player profiles, etc. To describe 
the components and operation of the present invention, the 
production and transmission of a live sporting event 10, i.e., 
football, is chosen to present the invention features. How- 
ever, other applications can be appreciated by the reader, 
including several disclosed below. 

[0030] A, The Control Studio 

[0031] The interactive broadcast program is prepared at 
the control studio 5 into digital packages. The control studio 
5, as shown in FIG. 1, allows a producer to create and 
introduce interactive elements during a live broadcast. In 
one preferred embodiment, the producer pre-records a set of 
interrogatories or instructions for the user. These interroga- 
tories may include such questions as the following: 

[0032] SELECT THE CAMERA ANGLES/OPTIONS 
THAT YOU PREFER: 

[0033] OPTIONS: END-ZONE 

[0034] FIFTY YARD LINE 

[0035] FOCUS ON THE QUARTERBACK 

[0036] FOCUS ON THE DEFENSE 

[0037] FOCUS ON THE CHEERLEADERS 

[0038] PROVIDE HIGHLIGHTS 

[0039] WHO IS YOUR FAVORITE TEAM IN THE 
GAME? 

[0040] OPTIONS: BLUE TEAM 
[0041] RED TEAM 

[0042] WHO IS YOUR FAVORITE PLAYER IN THE 
GAME? 

[0043] OPTIONS: RED RUNNER 
[0044] BLUE QUARTERBACK 
[0045] BLUE DEFENSIVE END 
[0046] RED QUARTERBACK 
[0047] ETC. 

[0048] ARE YOU INTERESTED IN RECEIVING 
STATISTICAL SUMMARIES? 

[0049] WOULD YOU ENJOY LISTENING TO 
PLAYER INTERVIEWS DURING THE GAME? 

[0050] Such interrogatories can be presented to the viewer 
at the beginning of the broadcast or scattered throughout the 



program. Interactive responses to such interrogatories would 
include video, audio and graphics personalized to the par- 
ticular viewer. The preparation of the graphics for presenting 
such interrogatories occurs off-line at the control studio 5 
using chyron or any graphics language. These interrogato- 
ries or instructions will ultimately be displayed to the home 
viewer, preferably in the form of graphics, to facilitate the 
interactive responses. The producer creates these graphic 
video slides of questions on a computer using the text editor 
and chyron. Associated with each question, the producer 
enters a number of possible viewer options. Then, the 
producer relates each possible user entry to one or more 
corresponding interactive responses. If the response is infor- 
mation from an Internet Web page, the producer will indicate 
the particular Uniform Resource Locator (URL) of the Web 
page. The producer sets a time stamp for when each par- 
ticular question will appear during the program. As 
explained below in more detail, the viewer response to a 
query will be used to direct which video (and/or audio, 
graphics, or Web page) option will be provided to the viewer. 
In the present invention, the interactive response to the query 
can occur immediately following the entry of the viewer 
entry or at some predetermined later time in the program 
using "trigger points,"500 as explained in detail below. 

[0051] With respect to the video segments, cameras 100 
are preferably trained on different segments of the sporting 
event 10. As is common with broadcasts of a football game, 
for example, cameras 100 could be located in the endzone, 
press box, the field and at various other locations throughout 
the stadium. Further, various video options can be created 
including video replay, slow motion, isolation on cheerlead- 
ers, particular player or group of players. Instant replays are 
created by delaying the live feed for a certain number of 
seconds. These video streams are sent to a control studio 5. 
The control studio 5 contains the necessary equipment for 
packaging the program for delivery to the viewers. The 
studio 5 contains a video switcher 105 which receives the 
live signals from the cameras 100 by way of various input 
lines. Further, lines carrying recorded video streams from 
one or more VCRs 110 , computers or CD players feed into 
the video switcher 105. The video switcher 105 also receives ' 
video inputs from the control computer 135 . Further, various 
graphics screens, depicting, for example, sports team or 
playe'r statistics can be designed with the control personal N 
computer 135 and forwarded to the digital video switch 105. 
The producer, via the control PC 135, directs which video 
options to pass through the video switcher 105. At the output 
of the video switcher 105, each of the different output video 
streams access a separate encoder 125 and are all GEN- 
LOCKED, so that each video stream is synchronized with 
the other video streams. 

[0052] After encoding, the video streams are input into a 
video compressor 125. Preferably, the digital compression 
scheme is MPEG -2. Preferably, 64 Quadrature Amplitude 
Modulation (QAM) is used as the modulation scheme. In 
this digital embodiment, four channels of digitally-com- 
pressed video content would carry about 27 Mbps using 6 
MHz of bandwidth. Alternatively, if 256 QAM is employed, 
seven-to-one video compression can be achieved with the 
MPEG-2 scheme. While MPEG-2 is the preferred compres- 
sion scheme, the signals can be compressed according to any 



07/08/2004, EAST Version: 1.4.1 



US 2002/0188943 Al Dec. 12, 2002 



known standard including MPEG-1, JPEG, or other DCT 
coding scheme, wavelets, fractals or other transform or 
waveform based technique. 

[0053] The control studio 5 also contains an audio 
switcher 115 which receives live audio signals from micro- 
phones or recorded audio from tape players 120, CDs, VCRs 
110, etc.. The control computer 135 sends commands to the 
audio switcher 115 directing which audio options should 
pass through the switch 115. Further, in the audio switcher 
115 the various audio signals can be aligned to match the 
various video signals in time. In addition, VCR audio output 
is received by the audio switcher 115. The present invention 
can accommodate any number of audio signals as output 
from the audio switcher 115, as directed by the producer. 
The audio outputs are received by an digital audio encoder/ 
compressor 130. The audio signals arc then preferably 
sampled, encoded and compressed in the digital audio 
encoder/compressor 130. The encoding technique can be a 
waveform coding technique such as PCM, ADPCM or DM. 
Alternatively, the signals can be encoded using synthesizer 
or vocoder techniques such as MUSICAM, Linear Predic- 
tive Coding (LPC), Adaptive Predictive Coding (APC), and 
Sub-band coding. Generally, the transmission rate is about 
256 kbps per audio for the stereo pair. 

[0054] The timing and control for integrating the various 
multimedia elements is provided by the ACTV authoring 
language, a unique set of interactive data codes to facilitate 
the interactive process. The data codes are stored in memory 
in the control computer 135 as part of the ACTV program- 
ming language. The codes comprise commands, or branch 
codes, for branching between interactive options, timing 
signals for controlling the interactive program, data or text, 
commands for termination and initiation or interactive pro- 
gram viewing, or triggers for executing macros. Preferably, 
thesejxunmaridsi are output from the control computer 135 
an^^uUiplejce3l>vith the video streams in the MPEG-2 
compressor 125, as shown in FIG. 1. Interactive options that 
can be branched to based on the branch codes include video 
segments, audio segments, graphics segments and/or iden- 
tified Web pages. 

[0055] There are several commands in the ACTV author- 
ing language that function to provide the interactive sports 
programming applications. These commands are explained 
below in Section D, which details programming applica- 
tions. 

[0056] B. The Transmission System 

[0057] As shown in FIG. 1, the digital interactive system 
1 uses an interactive program delivery system with any 
transmission means including satellite 15, cable 150, wire or 
television broadcast 175 to deliver the interactive program 
(hereinafter "composite interactive program") from the con- 
trol studio 5 for distribution to subscribers in their homes. At 
the control studio 5, the signals from the digital multiplexer 
140 are converted to RF and distributed to a microwave 175, 
cable 150 or satellite 15 network. Preferably, the digital 
interactive signal is forwarded from the control studio 5 to 
a cable headend 150, and subsequently, sent to the homes via 
the cable network. 

[0058] The program is preferably the broadcast of a live 
event. For example, live sporting events with added inter- 
active elements can be broadcast from the control studio 5. 



Such live interactive elements could be different camera 
angles 100, slow motion video, etc., as discussed above, 
while also incorporating prerecorded interactive segments 
such as highlights. Alternatively, the program can be pro- 
duced off-line and stored in a program storage means at the 
control studio 5. 

[0059] In a satellite broadcast transmission, the digital 
interactive signals are transmitted to uplink equipment 
where they may be multiplexed, upconverted, modulated, 
amplified and transmitted by satellite 15 to the receiver site 
155 for distribution to the homes. 

[0060] At the reception end, the composite digital inter- 
active signals enter a receiver 155 where the signals are 
demultiplexed, downconverted, demodulated and then 
passed to a cable distribution system that directs the signals 
to the homes. Although a cable distribution system 150 is the 
preferred transmission media to the homes, the digital sig- 
nals may also be distributed by any conventionally known 
technique including satellite 15 to digital satellite receivers 
155 at the home, fiberoptics, low or high power broadcast 
television 175, telephone lines, cellular networks, and simi- 
lar technology can be used interchangeably with this pro- 
gram delivery system. 

[0061] C. The Interactive Digital Box 

[0062] The interactive digital box 25 is shown schemati- 
cally in FIG. 2. Preferably, the interactive digital box is a 
specially adapted digital cable box 25. The controller 260 
determines what video, audio, graphics and/or Web pages to 
display based upon the interactive commands which it 
receives. Based upon the commands, it plays the appropriate 
video, audio, graphics or Web page options. The graphics 
can either be created and sent from the control studio 5 or the 
graphical images can be created at the interactive digital box 
25 based on instructions preferably in the interactive com- 
mands. The interactive digital box 25 connects to a televi- 
sion 165 or other display monitor. Further, the interactive 
digital box 25 can be connected to a digital television 195, 
in which case an RF modulator 245 is not necessary. Each 
downstream transmission reaches the subscriber's house, 
shown in FIG. 2, preferably through a tap and drop cable. 

[0063] The user interacts with the program through the 
input device 20. Preferably, the input device 20 is a typical 
television remote. The user interface 270 may be an infrared, 
wireless, or wired receiver that receives information from 
the input device 20. 

[0064] Regardless of the type of input device 20, user 
inputs can be utilized by the present invention immediately, 
or at a later time, to result in personalized graphics, video 
and/or audio presentation. For example, the present inven- 
tion utilizes "trigger points,"500 as described below, to 
enable subsequent branches among multimedia segments 
during the show. 

[0065] Certain commands are sent from the control studio 
5 as part of the digital interactive programming to facilitate 
the collection of user entries. These commands are extracted 
at the digital demultiplexer 210 and sent to the controller 260 
which performs the appropriate action based on the com- 
mands. Some of these commands are explained below. 
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[0066] a. BEGIN INPUT EXTENDED 

[0067] The Begin Input command starts an input period 
during which the user may press one or more buttons to 
select his or her choices). The entry format of this command 
is set forth as follows: 

[0068] BEGIN INPUT {{N}{VALID 

KEYS } { DIFFERENT} { FEEDBACK} where: 



N The maximum number of keys that can be pressed. 

KEYS The valid keys that can be pressed. 

DIFFERENT Requires each key pressed to be different. 
FEEDBACK The feedback type provided to the viewer. 



[0069] b. BEGIN VIDEO CHOICE EXTENDED 

[0070] The Begin Video Choice begins an input period for 
disco mode. During disco mode, the video switches dynami- 
cally each time the viewer makes a selection. 

[0071] BEGINVIDEO CHOICE MODE [KEYS] 
{FEEDBACK [AUDIO]}" 

[0072] The disco mode allows the viewer to change chan- 
nels at will, while the OneShot mode allows only one change 
of channel. 

[0073] MODEDISCO/ONESHOT 

[0074] KEYS The valid keys that can be pressed. 

[0075] FEEDBACK The feedback supplied to the 
user for the key(s) which are pressed. 

[0076] The mode Disco allows the viewer to change 
channels at will, while the OneShot mode allows only one 
change of channel. 

[0077] C BEGINAUDIO CHOICE EXTENDED 

[0078] The Begin Audio Choice begins an input period for 
disco mode. 

[0079] BEGINAUDIO_CHOICE MODE [KEYS] 
{FEEDBACK [AUDIO]} 

[0080] MODEDISCO/ONESHOT 

[0081] KEYS The valid keys that can be pressed. 

[0082] FEEDBACK The feedback supplied to the 
user for the key(s) which are pressed. 

[0083] d. MAP 

[0084] The Map command is used to map video or audio 
tracks to keys, for use in connection with the Begin Audio 
Choice and Begin Video Choice commands. If this com- 
mand is omitted, Key 1 will map to Track or Channel 1, Key 
2 to Track or Channel 2 etc. This command allows mapping 
any key to any channel. 

[0085] MAP KEYS [TRACKS [Tl, . . . TN/T1-TN] 
CHANNELS [CI, . . . CN/C1-CN]] 

[0086] The map statement maps audio tracks or video 
channels to keys, to enable audio or video choice commands 
to effect changes to tracks other than the default tracks, 
which are that key 1 maps to track 1, key 2 to track 2, etc.. 
All the choice statements after a map statement will cause 
the tracks, video tracks or channels to be changed to those 



specified in the map statement. The acceptable numbers for 
the map command are from 1 to 8, for keys, video channels 
and audio channels. 

[0087] Other commands include those which allow for the 
following applications: (1) viewer profiling, to enable the set 
top box 25 to "remember" viewer preferences; (2) uploading 
viewer responses to a central location; (3) downloading of 
text and graphics, for display using the graphics chip of the 
set top box 25; (4) the ability of the viewer to prepare his 
own video, based upon his selections of camera shots 100 
and audio, which can be stored and replayed for the viewer. 

[0088] The interactive digital box 25 of the present inven- 
tion enables seamless flicker-free transparent switching 
between the digital video signals. "Seamless" means that the 
switch from one video signal to another is user impercep- 
tible. Because the video signals are running off the same 
clock, the interactive digital box 25 is capable of providing 
a seamless digital switch from one video signal to another 
signal. The program clock reference necessary for the box to 
make this seamless switch is preferably embedded in the 
signal header. 

[0089] As shown in FIG. 2, a CPU 260 is connected to an 
RF demodulator 200 and digital demultiplexer 210. The 
CPU 260 directs demodulation and demultiplexing of the 
proper channel and data stream to obtain the correct video 
signal. Seamless switching can occur with MPEG-2 com- 
pressed signals since there are points within the frame 
wherein seamless switching can occur. Preferably, switches 
occur at an "I" frame, assuming the use of MPEG-2 com- 
pression. The selected video signal is determined either by 
examination of the user's input from user interface 270 
and/or any other information or criteria (such as personal 
profile information) stored in RAM/ROM 265. For example, 
the RAM/ROM 265 could store commands provided within 
the video signals as discussed in U.S. Pat. No. 4,602,279, 
and incorporated herein by reference. 

[0090] The RF demodulator 200 demodulates data from 
the broadcast channel directed by the controller 260. After 
the data stream is demodulated, it passes through a forward 
error correction circuit 205 into a digital demultiplexer 210. 
The demultiplexer 210 is controlled by the controller 260 to 
provide a specific video signal out of a number of video 
signals which may be located within the data stream on the 
demodulated broadcast channel. The demultiplexed video 
signal is then decompressed and decoded by decompressor/ 
decoder 215. The video signal is synchronized by a sync add 
circuit 220 and a sync generator 225. The video signal is 
then buffered by a video frame buffer 230. The buffered 
video signal is modulated by a modulator 245 into a NTSC 
compatible signal. Such a modulator is not necessary if the 
selected signal is sent to a digital television 195. 

[0091] By using a video frame buffer 230 and delaying the 
viewing of a given signal, enough time is allowed for the 
decompressor/decoder 215 to lock onto, decompress, con- 
vert to analog, and wait for the resultant vertical interval of 
a second video signal. For example, assume video signal A 
is currently being processed and transferred through the 
circuit shown in FIG. 2 and displayed. Based upon a user 
selection, the controller 260 directs the digital demultiplexer 
210 and RF demodulator 200 to switch to another video 
signal, video signal B. To accomplish this, the analog video 
from the first digital video signal, video signal A, complete 
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with video sync, is fed into video frame buffer 230. This 
buffer 230 can hold the full video picture for "n" number of 
frames after which the signal is output to the display. In 
effect, a delayed video signal A is viewed "n" number of 
frames after the signal has been received. When the user 
selects a different video path by means of pressing a button 
on a keypad or entry by other means, the controller 260 
instructs the digital demultiplexer 210 to stop decoding 
signal A and lock onto signal B to begin decoding signal B 
instead of signal A. 

[0092] While this is happening, even though the decom- 
pressor/decoder 215 is no longer decompressing video sig- 
nal A, the display is still showing video signal A because it 
is being read from the buffer 230. As soon as decompressing 
and decoding occurs, the controller 260 looks for the next 
vertical blanking interval (VBI) and instructs the video 
frame buffer 230 to switch to its input, rather than its 
buffered output at the occurrence of the VBI. 

[0093] Since the RF demodulator 200, forward error cor- 
rector 205, digital demultiplexer 210, and decompressor/ 
decoder 215 require a certain time period to decompress and 
decode the video signal B frame from its data stream, the 
size of the buffer 230 has to be large enough so that this 
processing can take place without interruption during the 
switching of the video signals. If desired, the system may 
continue to use the buffer in anticipation of a future switch. 
By using the controller 260 to manipulate the fill and empty 
rate of the buffer 230, the buffer 230 may be rapidly filled 
with video signal B frames and then after a period of time 
will be reset and ready to make another switch to another 
video in the same manner. The buffer 230 may also be reset 
by skipping frames or providing a delay between sequential 
frame outputs for a short time in order to fill the buffer 230. 
If a delay is used to maintain video signal or frame output 
while the buffer 230 is being filled, a slight distortion may 
occur for a brief amount of time. 

[0094] Because a first video signal is always displayed as 
the output of the buffer 230 after the delay, the buffered 
video masks the acquisition and decoding of a second video 
signal. As long as the buffer 230 is large enough to keep the 
first video running while the second video is being decom- 
pressed and decoded, a seamless switch will occur. 

[0095] While the digital interactive box 25 of FIG. 2 
provides video interactivity, audio and/or graphics interac- 
tivity is also provided. For example, if, based on the viewer 
profile or viewer response to query, it is determined that the 
viewer's primary language is Spanish, then that viewer 
could obtain Spanish commentary to the football, soccer, 
etc. game. Alternatively, if a viewer has a favorite athlete, 
the audio can switch to an interview with the athlete during 
a segment of the broadcast. Multiple digital audio options 
forming a set of suitable responses to an interrogatory 
message can be sent as part of the composite digital signal. 
As set forth in U.S. Pat. No. 5,585,858, herein incorporated 
by reference, there are a number of different ways to 
effectively forward the necessary audio options for a given 
live event to the digital interactive box 25. With the present 
invention, it makes no difference how the audio options 
reach the digital interactive box 25, as long as they are 
available for selection and play at the appropriate times. 

[0096] In FIG. 2, the digital demultiplexer 210 extracts 
the digital audio signal(s) and forwards them to the audio 



switch 250. Additional audio options are available from the 
digital audio memory 255. At certain times during the 
program, the data codes will identify the selection of a 
particular audio option corresponding to previous user 
inputs. The controller 260 calls the appropriate audio options 
from internal memory 255 or directs the audio switch 250 to 
select a predetermined audio segment received as part of the 
received digital signal for passage to the RF modulator 245 
for play to the subscriber. At the end of the audio segment 
time period as indicated by the data codes, the controller 260 
instructs the audio switch 250 to again pick up standard 
audio. 

[0097] The digital demultiplexer 210 sends the extracted 
graphics data or ACTV data codes to the controller 260. The 
controller 260 interprets the extracted data as either control 
data, including instructions for switching between video 
signals, audio signals, or graphics data for on-screen display. 
If the data is on-screen display data, the data is preferably 
prefixed by a command designating the data as on-screen 
display data, as opposed to control data. Further, the con- 
troller 260 also examines the control data for the occurrence 
of a header code designating the onset of a trigger point 500 
in the program, explained below. 

[0098] FIG. 3 shows an alternate, dual tuner embodiment 
for seamless switching between separate video signals. This 
embodiment presumes that two 6 MHz channels are used, 
each of which comprises compressed digital video and audio 
streams. In this embodiment, the microprocessor 260 con- 
trols the selection of the RF channel that is demodulated by 
RF demodulatois 200A, 200B. The demodulated data 
streams enter the forward error correctors 205A, 205B. At 
the output of the forward error correctors 205A, 205B the 
data streams are transmitted to the input of the digital 
demultiplexers 210 A, 210B. 

[0099] As with the RF demodulators 200A, 200B, the 
digital demultiplexers 210A, 210B are controlled by the 
microprocessor 260. This configuration allows the micro- 
processor 260 to independently select two different indi- 
vidual time-multiplexed video signals on different channels 
and data streams. If all the video signals of an interactive 
program were contained on a single channel or data stream, 
it would only be necessary to have a single RF demodulator 
200, forward error corrector 205, and digital demultiplexer 
210 serially connected and feeding into the two digital video 
buffers 230A, 230B. 

[0100] Two data streams are provided from the digital 
demultiplexers 210A, 210B. One data stream carries video 
information pertaining to the video signal the user is cur- 
rently viewing. The second data stream carries the video 
signal selected based on the user's previous and/or current 
interactive selections from the user interface 270, as deter- 
mined by the microprocessor 260. 

[0101] The digital information on each of the two streams 
is buffered in digital video buffers 230A, 230B. The buffered 
signals are then decompressed and converted into analog 
signals by decompressors/decoders 215A, 215B which 
include digital to analog converters. The decompressors 
215A, 215B are preferably MPEG-2 decoders. 

[0102] A local sync generator 225 is connected to sync add 
220A, 220B and frame sync 380A, 380B circuits. Because 
both streams are synchronized based on signals from the 
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same local sync generator 225, each stream becomes syn- 
chronized to the other. In particular, the signals on each 
stream are frame synchronized. 

[0103] A vertical blanking interval (VBI) switch 335 is 
connected to the microprocessor 260 so that the input may 
be switched during the vertical blanking interval of the 
current stream, resulting in a seamless switch to the viewer. 

[0104] The embodiment of FIG. 3 operates as follows. 
Based on user responses and control codes, it is assumed that 
the microprocessor 260 determines that a switch from video 
signal A to video signal C should be performed. The RF 
demodulator 200A and digital demultiplexer 210A are pro- 
cessing the currently viewed video signal, video signal A, 
which is progressing through the upper branch components. 
A command is issued from the microprocessor 260 to the RF 
demodulator 200A, 200B commanding a switch to the 
channel and data stream on which video signal C is located. 
The microprocessor 260 also instructs the digital demulti- 
plexer 210B to provide video signal C from the received data 
stream to digital video buffer 230B. 

[0105] At this point, the upper RF demodulator 200A and 
digital demultiplexer 210A are still independently receiving 
and processing video signal A, which continues through the 
upper branch of the circuit. 

[0106] At a certain point, the digital decompressor/de- 
coder 215B in the lower branch will begin filling up with 
video signal C frames. After video signal C is decompressed 
and decoded, it is converted into analog. A local sync 
generator 225 inserts both local sync and frame sync to 
video signal C via sync add circuit 220B and frame sync 
circuit 380B in order to synchronize it with the currently 
displayed video signal A, which is still being provided from 
the upper digital video buffer 230A. At the appropriate 
switch point, triggered by programming codes supplied with 
each video signal A and C, the microprocessor 260 directs 
the VBI switch 335 to switch in the vertical blanking interval 
from video A to video C, at which time video C will then 
seamlessly appear on the computer screen. 

[0107] Digital video buffers 230A, 230B maybe used in 
the circuit of FIG. 3, but are optional. However, in an 
alternative embodiment the buffers would be required to 
provide a seamless switch if the FIG. 3 circuit was modified 
to incorporate a single RF demodulator 200, single forward 
error corrector 205, and single digital demultiplexer 210, 
each with a single input and single output. In this alternative 
embodiment, the circuit cannot independently receive and 
demultiplex two data streams on different frequency chan- 
nels. One buffer 230A is used to store previously received 
video signals, while the other buffer 230B quickly passes 
through the selected video signals. 

[0108] Based on the same assumptions above, video signal 
A is progressing through the upper branch of the circuit and 
it is desired to switch to video signal C. However, in this 
alternative embodiment, the digital video buffer 230A is 
providing maximum buffering to video signal A. 

[0109] Because it is desired to switch to video signal C, 
the microprocessor 260 directs the alternative circuit (con- 
taining a single RF receiver 200, single forward error 
corrector 205 and single digital demultiplexer 210 connected 
in serial), to receive and demultiplex the data stream on 
which video signal C is located, which may be different than 



that of video signal A. When video signal C is demulti- 
plexed, the microprocessor 260 directs the digital video 
buffer 230 to provide minimum buffering of video signal C 
so that decompressor/decoder 215 may quickly decompress 
and decode the digital signals. After decompression and 
decoding, video signal C is synchronized with video signal 
A. At this time, video signal A is read for display from digital 
video buffer 230A. The upper digital video buffer 230Amust 
be large enough to provide video frames for output during 
the time it takes the RF demodulator 200 and digital demul- 
tiplexer 210 to switch to video signal C and the time required 
for decompression, decoding, and synchronization of video 
signal C. 

[0110] When video signal C is synchronized with video 
signal A, the microprocessor 260 directs VBI switch 335 to 
switch from video signal A to video signal C in the vertical 
blanking interval of video signal A, thereby providing a 
seamless and flicker-free switch. 

[0111] At this time, digital video buffer 230 will begin to 
utilize maximum buffering by altering its fill/empty rate as 
described above with respect to the FIG. 3 embodiment. 
When adequate buffering is achieved, a switch to another 
video signal may be performed in the same manner as 
described above. 

[0112] Another preferred embodiment is shown in FIG. 4. 
This embodiment also includes an RF demodulator 200, a 
forward error corrector 205, and a digital demultiplexer 210. 
However, the circuitry differs along the rest of the chain to 
the television set or monitor. In this embodiment, a memory 
475 is incorporated and connected to the output of the 
demultiplexer for storing the compressed composite digital 
video signal. The decompressor/decoder 215 is inserted at 
the output of the compressed memory. The decompressor/ 
decoder 215 decompresses the digital signal, converts the 
signal to analog and forwards the analog signal to the RF 
encode 245 for transmission to the monitor. Once the 
composite compressed digital video signal is fed into the 
compressed memory 475, the microprocessor 260 directs a 
pointer to be placed somewhere along the compressed 
digital video signal. Based on the placement of the pointer, 
different frames and different segments of the composite 
digital video signal will be read from memory 475 for 
decompression and decoding. 

[0113] The different video signals are distinguished from 
one another because they are labeled, preferably by headers. 
Assuming that video signal A has been selected for play on 
the monitor, the compressed digital memory 475 fills up with 
A frames. Assuming a switch to video signal C is desired, the 
microprocessor 260 directs the RF demodulator 200 and 
digital demultiplexer 210 to begin filling the compressed 
memory 475 with video C frames. The decoder 215 pointer 
begins to move down. As soon as a sufficient number of C 
frames have entered the compressed memory 475, the 
pointer will then jump to the beginning of the C frames. The 
C frames are then output into the decompressor/decoder 215 
where the digital frames are converted into an analog signal. 

[0114] The digital video is multiplexed in a series of easily 
identifiable packets. These packets may contain full com- 
pressed frames of video (I frames) or may include only the 
differences between full frames (B frames or P frames). 

[0115] To be able to reconstruct the full video images, the 
decompressor/decoder 215 needs to have a minimum num- 
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ber of I, P and B frames. The decoder 215 needs only one I 
frame to decode an image. Conversely, two prior Anchor 
frames ("I's" and "P's") are necessary to decode B frames. 
In order to decode P frames, the decoder 215 only needs one 
Prior Anchor frame. When the microprocessor instructs the 
digital demultiplexer 210 to start sending packets from a 
different data stream there is no way to be certain that the 
next packet will be an I packet needed for decoding the 
second video stream. To avoid a breakup of the video 
images, which would occur if the decompressor/decoder 215 
suddenly started receiving packets unrelated to the stream it 
was decoding, the microprocessor 260 starts to fill up the 
memory 475 with video signal C packets until it is deter- 
mined that a full sequence of I, B and P frames are available. 
The decoder 215 should receive the last bit of the last B 
frame in a given, GOP (Group of Pictures) before the switch, 
in order to prevent glitches when decoding. Furthermore, the 
last B frame of the GOP must only be backward predicted, 
not forward predicted or bidirectional predicted. As soon as 
the valid sequence is in memory 475 the microprocessor 260 
moves the memory read pointer to the start of a valid 
sequence of C video signal packets so that the decompres- 
sor/decoder 215 can successfully decode the C signals. This 
results in a seamless switch from video signal A to video 
signal C. 

[0116] This embodiment requires a data channel for 
enabling a synchronous switch between a first video stream 
and a second video stream. This data channel comprises the 
ACTV codes which link together the different program 
elements and information segments on the different video 
signals. In addition, the data channel also comprises syn- 
chronization pulses and a time code to signify to the pointer 
the proper time to skip from a memory location representing 
one video signal to a memory location representing another 
video signal in order to enable a seamless switch. 

[0117] The microprocessor 260 reads the data signal from 
the digital demultiplexer 210 and communicates pertinent 
data to the sync add circuit 220, which is connected to sync 
generator 225. The microprocessor 260 is then able to 
synchronously communicate with the memory 475. 

[0118] The time code sent will identify the timing for one 
picture, as well as for multiple pictures, and will lock the 
different pictures together. This is done through the use of 
similar clocks at both the transmission end and the receiver. 
A time code is used in order to keep the two clocks at both 
the transmission and receive end synchronously connected 
to one another. Once the clocks at both ends are working 
synchronously, each of the multiplexed video streams must 
be synchronized to the clocks. In order to synchronize the 
multiplexed video stream to the clocks, each of the indi- 
vidual channels must be referenced to a common reference 
point and must be identified. 

[0119] In the preferred embodiment, a packet header 
would be incorporated into the transport layer of the MPEG 
signal to identify the various channels. The packet header 
will also include information as to where to insert the 
vertical blanking interval. In MPEG, the vertical blanking 
interval is not transmitted from the headend. Therefore, the 
vertical blanking interval must be generated locally. The 
packet header eye will identify at what time the vertical 
blanking interval is in existence in order to effectuate a 
seamless switch between analog pictures. 



[0120] In summary, the combination of clock and the 
information embedded in either the transport layer of MPEG 
or in a separate packet on a separate data channel effectuates 
the linking between each video signal and a corresponding 
time point. The data channel also includes information 
designating when all the various video signals will be in 
synchronism with one another. It is at these points that the 
microprocessor 260 may direct the pointer to skip from one 
location to another location, at a time (such as during the 
VBI) when a seamless switch will result. 

[0121] D. Trigger Points 

[0122] Interactivity is further enhanced in the digital inter- 
active embodiments through the application of trigger points 
500 scattered at various predetermined times throughout the 
program, a timeline representation of which is shown in 
FIG. 5. The trigger points 500 correspond to times when 
interactive events are scheduled to take place during the live 
sporting event 10. These interactive events could be the 
selection and playing of video, audio segments, the display 
of graphics or display of Web pages accessed from Internet 
Web sites 170. For example, when a viewer's favorite 
baseball player is at bat, graphics showing past or current 
performance statistics of the player can be overlaid on the 
screen while excerpts from an interview with the player can 
be played for the viewer. While the choice of particular 
video, audio or graphics is still dependent on viewer selec- 
tions, the viewer selections in response to displayed graphi- 
cal interrogatory messages are preferably made during a 
period at the onset of the program or when a viewer first 
tunes into the program. Alternatively, interrogatories are not 
necessary if the switches are based on the viewer profile 
stored in memory 265. These viewer selections are then 
utilized as inputs to macros called up at later times during 
the program by the controller 260 upon the occurrence of the 
trigger points 500, identified to the interactive computer by 
unique codes embedded in the video signal. 

[0123] The trigger points 500 correspond to the times 
when the conventional program content can be altered and 
personalized for the viewers. The programmer can place the 
trigger points 500 at any time throughout the program. Since 
the trigger points 500 are unknown to the subscriber, the 
subscriber does not know when they will receive a person- 
alized message. In other words, an interactive response can 
either immediately follow a corresponding user selection 
made to an interrogatory message or occur at a later time 
corresponding to a trigger point 500, or any combination of 
the two. Of course, timing of the interactive events should 
correspond to suitable times in the program where branching 
to interactive elements is sensible and does not clash with 
the program content of the conventional video still displayed 
on the television 165 or other display monitor. 

[0124] At the onset of a trigger point 500, the controller 
260 will select one of several possible audio (or video or 
graphic display) responses for presentation to the subscriber. 
As mentioned above and shown in FIG. 5, some of the 
responses may comprise a branch to either a video segment, 
graphics and/or audio segments. 

[0125] In combination with the use of trigger points 500, 
the present invention allows for the viewer to select certain 
options at the onset of the program to suit the viewers' 
preferences. For example, if the program broadcast is a live 
sports event 10, at an early trigger point 500, the viewer 
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could be queried as to whether the viewer would prefer to 
receive audio in English, Spanish, French, or perhaps hear 
the local announcer instead of the network announcer. Upon 
the viewer selection, the CPU 260 directs a branch to the 
appropriate interactive segment. 

[0126] Each trigger point 500 is identified preferably 
through the broadcast of ACTV codes sent as part of the 
composite interactive program signal The codes preferably 
include, at a minimum, the following information: (1) 
header identifying the occurrence of a trigger point 500; (2) 
function ID (e.g., selection of audio or graphics responses, 
etc.); and (3) corresponding interrogatory message(s) or 
particular viewer characteristic or habit based on viewer 
profile. The first bit sequence simply identifies to the con- 
troller that a trigger point 500 is about to occur. The function 
ID designates the macro or other set of executable instruc- 
tions for the controller 260 to read and interpret to obtain the 
desired result, e.g., a selected video and/or audio response. 

[0127] Upon extraction of the codes by the data decoder, 
the controller 260 reads and interprets the codes and calls 
from memory 265 a particular user selection(s) designated 
by the trigger point 500 codes. The user selections corre- 
spond to subscriber answers to a series of interrogatory 
messages preferably presented at the beginning of the pro- 
gram. After obtaining the appropriate user selection(s), the 
controller 260 reads and performs the executable instruc- 
tions using the user selections) as input(s) in the macro 
algorithm. The result of the algorithm is either a selected 
video stream, audio and/or selected graphics response. The 
video/audio response can be called from memory 265 if it is 
prestored, called from external data storage, or the controller 
260 can command the switch to branch to the particular 
video audio stream if the response is broadcast concurrently 
with the trigger point 500. After the selected video/audio 
response is played to the subscriber, the switch branches 
back to the standard program, shown at time t 8 in FIG. 5. 

[0128] As mentioned above, a series of interrogatory mes- 
sages are preferably presented when the subscriber begins 
watching the interactive program. These interrogatory mes- 
sages can be presented in any one of three ways. First, the 
interrogatory messages can be presented as graphics dis- 
plays overlaid by the interactive computer workstation onto 
a video signal, wherein the graphics data is sent in the 
vertical blanking interval of the composite interactive signal, 
or alternatively stored on the hard disk or external storage. 
Second, the interrogatory messages are presented as graph- 
ics displays as discussed above, except the graphics data 
comes from local storage, external data storage (e.g., CD 
ROM, cartridge, etc.), or a combination of data in the VBI 
and data called from either local or external data storage. 
Third, graphics data can be presented in the form of user 
templates stored at the interactive computer workstation. 

[0129] User selections corresponding to answers to the n 
successive interrogatory messages are received by the 
remote interface 270 at the beginning of the show, stored in 
memory 265 and used throughout the show at the appropri- 
ate trigger points 500 to subtlety change program content as 
the show progresses. Preferably, each interrogatory has a set 
of possible answers. Next to each possible answer will be 
some identifier corresponding to a label on a key on the user 
interface. The subscriber depresses the key corresponding to 
their answer selection. This selection is decoded by the 



remote interface 270 and controller 260, stored in memory 
265, preferably RAM, and used later as required by an 
algorithm designated at a trigger point 500. 

[0130] E. Internet 

[0131] In addition to the central studio serving as a source 
of interactive option responses, the Internet can be used as 
a source of personalized information for interactive 
responses. 

[0132] As discussed above, the video programming is 
preferably created at a centralized location, i.e., the control 
studio 5 as shown in FIG. 1, for distribution to subscribers 
in their homes. Referring to FIGS. 1 and 6, in a preferred 
method, the operator at the control studio 5 must designate 
certain Web pages to correspond to one of the program 
options, such as audio and/or graphics options, using control 
PC 135 and URL encoder 600. Instead of encoding the 
actual content of the options at the control studio 5, as with 
the audio, Web address identifiers, i.e., Uniform Resource 
Locators (URLs) are encoded and sent as part of the data 
codes from the control PC 135 to the digital multiplexer 140. 
After multiplexing, the URLs are sent as part of the program 
signal 615, as described above. Preferably, the URLs, like 
the various audio and graphics options, have associated time 
stamps which indicate to the remote digital set top boxes 25 
when, during the video program, to display the particular 
Web pages addressed by the URLs, the selection" and display 
of which is preferably made as a function of viewer 
responses or viewer profile. 

[0133] Preferably, each digital set top box 25 has an 
Internet connection 160 created concurrently with the cable 
connection. The Internet connection 160 can be via high- 
speed line, RF, conventional modem. The digital set top box 
25 has Internet access 160 via any of the current ASCII 
software mechanisms. In a preferred embodiment, in the 
interactive digital set top box 25, the digital demultiplexer 
210 extracts the URLs along with the other data codes. In an 
alternative embodiment, a local URL decoder 605 at the user 
site extracts the URLs. 

[0134] In a preferred embodiment, a JAVA enabled 
browser as well as specialized software for performing part 
of the method of the present invention are installed on the 
interactive digital set top box 25. The JAVA enabled browser 
allows the interactive digital set top box 25 to retrieve the 
Web pages and is preferred software, since it is platform 
independent, and thus, enables efficient and flexible transfer 
of programs, images, etc., over the Internet. The specialized 
software acts as an interface between the video program- 
ming and the Internet functions of the present invention. The 
processor and software interprets these URLs and directs the 
JAVA enabled browser to retrieve the particular relevant 
Web pages, and synchronizes the retrieved Web pages to the 
video content for display on the television monitor 165 at the 
appropriate times. 

[0135] In the present invention, the viewer also has the 
capability to link to a channel website at will. For example, 
if a viewer is interested in purchasing a product described in 
an advertisement, by merely clicking on a button on their 
remote 20, the producer's Website could be accessed by 
Internet connection 160 and displayed to the viewer. The 
viewer could then either obtain more information about the 
product or order the product, if desired. As described above, 
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this applicatioD is possible by sending the URL associated 
with the producer's Website to the digital cable boxes 25 as 
part of the interactive program. Upon selection by the 
viewer, the web browser, located either in the digital set-top 
box 25 or externally in a connected PC 610, can retrieve the 
Web pages. The specialized software then synchronizes the 
Web pages for video display. 

[0136] F. Memory 

[0137] The interactive digital set top box 25 of the present 
invention also has the advantage of remembering subscriber 
responses and using these responses in choosing a video/ 
audio response, and/or graphics interrogatory message, to 
present to the student. Memory branching is a technique of 
the present invention where the algorithm assembles video/ 
audio responses and graphics interrogatory messages 
according to the current and previous user inputs. Memory 
branching is accomplished by linking video/audio streams 
and/or successive graphics interrogatory messages together 
in a logical relationship. In this scheme, the interactive 
digital set top box 25 contains logic (preferably, in the 
software algorithm) and memory 265 to store previous 
subscriber selections and to process these previous 
responses in the algorithm to control future video/audio 
stream selection, as well as future graphics message selec- 
tion. 

[0138] G. Digital Viewer Profiles 

[0139] In a preferred embodiment, the interactive digital 
cable box 25 can have a "viewer profile" stored in its 
memory 265. Alternatively, the accumulated profile of 
viewer characteristics and/or habits can be stored at the 
control studio 5 or cable headend. If the profile statistics are 
accumulated at some central location, they can also be sent 
to each viewer's home. On the other hand, if accumulated in 
memory 265 at each of the set top terminals 25, the data 
could be sent to the central location for storage and dissemi- 
nation by way of the digital back channel. Decisions regard- 
ing personalized advertising and viewing can then be made 
for a viewer or class of viewers based on the accumulated 
parameters. 

[0140] The "viewer profile" preferably contains charac- 
teristics of the particular viewer at that subscriber location, 
such as sex, hobbies, interests, etc. This viewer profile is 
created by having the viewer respond to a series of ques- 
tions. Alternatively, the viewer profiles could be created at a 
control studio 5 and sent to the interactive digital cable box 
25. This information is then used by the cable box software 
to create a compendium of the viewer's interests and pref- 
erences^ — i.e., a user profile. The stored user profile would be 
used in place of the question/answer format, and thus, 
dictate the branches to interactive segments of interest to the 
viewer. 

[0141] Alternatively, the interactive cable box 25 can be 
programmed to create a user profile of each viewer based on 
the selections made during one of the interactive programs. 
If the profile shows that a particular viewer does not enjoy 
violence, the system can automatically branch to another 
video signal at the commencement of a fight in a sports 
program. Furthermore, such a user profile could be modified 
or enriched over time based on selections made during 
future interactive programs. For example, the * memory' 
technique described above can be used to modify the user 
profile based on user response over time. 



[0142] Event data is collected from the viewer when the 
following command is received and processed by the con- 
troller 260 in the interactive digital cable box 25: 

[0143] VIEWER PROFILE 

[0144] This command is used to enable events which can 
be utilized for profiling during a show. The events indicated 
in this command are those profiling events which are pos- 
sible during the show. The enabled events may be either 
selected by the viewer during the show, or may be auto- 
matically enabled based upon viewer selections. 

[0145] VIEWER_PROFILE EVENT1, . . . EVENT24 

[0146] Once the profile is created, the programming 
choices or interactive responses can be triggered based on 
the content of the viewer profile itself. For example, if the 
viewer profile suggests that the viewer is particularly inter- 
ested in sports cars, a sports car commercial could be played 
for the viewer at a predetermined point in the program. As 
another application, if a viewer's profile indicates that the 
viewer is interested in cooking, whenever the viewer 
watches such a program, the user profile would trigger the 
interactive program to download recipes and either display 
such recipes on the screen or send the recipes to an attached 
printer. 

[0147] Viewer profile information can then be collected at 
the control studio 5 through polling of the viewer cable 
boxes 25 for viewer selection data. Special polling software 
is loaded into the digital set top box 25 for performing the 
polling functions. Alternatively, the invention allows for the 
digital set top boxes 25 to send back data on command from 
the control studio 5 or periodically. The command to initiate 
an upload of viewer profile data is as follows: 

[0148] UPLOAD EXTENDED 

[0149] This command is used to initiate the uploading of 
data to the central site. 

[0150] UPLOAD_EXTENDED UPLOAD IDENTI- 
FIER VARI AB LEN AM E 

[0151] [PHONE_NUMBER] 

[0152] UPLOADJDENTIFIERnumeric constant 
identifying upload. 

[0153] VARIABLE_NAME name of variable to be 
uploaded 

[0154] PHONE_NUMBER string, that consists of 
numbers only, it can be omitted, if not needed. 

[0155] Regardless of whether a polling or periodic scheme 
is used, the statistics and other user profile information is 
preferably sent back to the control studio 5 by use of the 
back-channel. 

[0156] H. Applications 

[0157] The embodiments, described above, allow for sev- 
eral possible applications. For example, in a live sports event 
10, one channel could carry the standard video channel, with 
other channels carrying different camera angles 100 and/or 
close-ups of particular players. Other potential video options 
include instant replay, highlights, player statistics via 
graphic overlays, etc. Graphics presenting statistical infor- 
mation on the players can be constantly updated using the 
Chyron system. 



07/08/2004, EAST version: 1.4.1 



US 2002/0188943 Al 



11 



Dec. 12, 2002 



[0158] The provision of trigger points 500, explained 
above, can provide for seamless integration of such video 
options during the sporting event, based on either the viewer 
responses to interrogatories at the beginning of the program 
and/or on the digital viewer profile. 

[0159] Further, the viewer can become the director with 
the present invention. For example, the viewer can choose 
which camera angles 100 to emphasize. In a broadcast of 
golf, the viewer can direct whether they desire to follow a 
particular player from hole-to-hole, focus on one particu- 
larly difficult hole. In this manner, the viewer can customize 
the sporting broadcast to meet his own interests. 

[0160] Further, the viewer can act as a director to create 
their own video. During a live concert 10, for example, the 
viewer selects various camera angles 100 at different times. 
These selections along with a time stamp, indicating the 
program time when each selection was made, are stored in 
memory 265. When the program is played back a second 
time, the processor 260 will automatically direct branching 
between the video channels according to the stored selec- 
tions at the time stamp. In this manner, a music video can be 
created by the viewer. 

[0161] The viewing experience can be further enlighten- 
ing for the viewer by implementing games and contests 
during the live sporting event 10. For example, graphics 
overlays can be developed that query the viewer during the 
game. During a football broadcast, for example, viewers can 
be queried with such interrogatories as the following: 

[0162] What will be the next play? (RUN/PASS/ 
KICK); 

[0163] Will the offense get the first down?; 

[0164] Will they score on this possession?; 

[0165] Pick the halftime score; 

[0166] Who will win? 

[0167] Each viewer's responses can be sent back to the 
control studio 5 for tabulation of scores. Preferably, the 
responses are packaged at the digital cable box 25 and 
transmitted to the control studio 5 via the digital backchan- 
nel upon the UPLOAD EXTENDED command. Alterna- 
tively, tabulation of scores can take place at the digital cable 
box 25 through the utilization of certain software in memory 
265. Each correct answer can correspond to a certain number 
of points. At the end of the game, the interactive program 
preferably presents a graphic showing the viewer point total. 
If desired, advertisers could present special gift certificates 
for excellent performance in such games. The provision of 
such certificates would occur by displaying a certain code 
that a viewer can take to a store to receive the gift. In this 
manner, viewer interests in sports events can be enhanced. 

[0168] Further, the viewer has the option with the present 
invention to block out viewing of certain events. For 
example, if the viewer is adverse to violence during a 
sporting event or other type program, the system can block 
out such options from the viewer with the following data 
commands: 

[0169] EVENT 

[0170] This command is used to indicate occurrence of a 
certain event (e.g., a fight breaking out during a football 
game). 



[0171] EVENT EVENTNUMBER 

[0172] EVENTJWMBER is a numeric constant. 
[0173] ON EVENT EXECUTE MACRO 

[0174] This command is used to implement certain actions 
(such as an automatic branch to another video channel in 
order to block out violent event, for example), as soon as the 
event arrives. 

[0175] ON EVENT EVENT NUMBER MACRO_N- 
UMBER 

[0176] EVENT_NUMBER is a numeric constant. 

[0177] MACRO_NUMBER is a numeric constant. 

[0178] I. Two-Way Configuration 

[0179] The live programming system of the present inven- 
tion may be operated in a two way configuration, as illus- 
trated in FIG. 7. In this mode, the various video signals are 
processed as previously described, being digitized and com- 
pressed at the control studio 5. The signals are then sent to 
a central switching station, or headend 30. 

[0180] In this embodiment, the switching between the 
various live digital signals is accomplished at the headend 
30 rather than at the receiver. On the receive end, each 
digital set-top box 760 relays viewer selections back to the 
remotely located switching station 30. Preferably, the viewer 
selections are relayed by way of the digital back channel 
770. However, the viewer selections may be relayed to the 
switching station 30 by any conventional means, such as 
two-way cable television, telephone or microwave transmis- 
sion. The switching station 30 receives the viewer selection 
and routes the desired signal to a transmitter 750 which 
conventionally transmits the desired video down the appro- 
priate digital cable channel for the particular viewer. 

[0181] At the central switching station 30, a demultiplexer 
710 demultiplexes the compressed signals and places each 
on a separate bus channel 725. A number of remote control 
interactive switches 730, 732, 734, 736 are connected to the 
video signal bus 725. Based on the viewer selections, an 
algorithm stored in memory 265 and under processor 260 
control at the central switching station 30, a digital seamless 
switch is made and the selected video, audio and/or graphics 
are forwarded to the viewer home for display. 

[0182] Such a two-way embodiment could be imple- 
mented in a video dial tone or video server system. In such 
a system, only a single video channel 755 is necessary for 
each home. Once the viewer selection is received at the 
server site at the cable headend 30, a switch is made to the 
appropriate video stream and this stream is sent on the single 
channel 755 to the home. 

[0183] Alternatively, it may be desirable to transmit an 
interactive sporting event over a single telephone line. When 
the viewer enters a selection on their remote 20, a signal is 
sent by way of the telephone line to the central switching 
station 30 which routes the desired signal of the interactive 
program over the user's telephone line so that a single link 
handles both the interactive choice being made at the 
receiver and the transmission of that choice from the hea- 
dend 30 where the actual switching takes place in response 
to the interactive selection made at the receiver. 
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[0184] The two-way link between the viewer and switch- 
ing station 30 may be used for other purposes. For example, 
demographic data may be transferred from the viewer to the 
broadcast network for commercial purposes, such as tar- 
geted advertising, billing, or other commercial or non- 
commercial purposes. 

[0185] While the present invention has been described 
primarily with respect to live events, and in particular 
sporting events, it has equal potential for enhancing content 
in other program categories. A viewer can become their own 
director of a murder mystery or other drama. By entering 
responses to displayed questions at the initiation of or during 
the show, the program will branch to alternative video/audio 
segments as a result of the user selections. In this manner, 
different viewers with different selections may end up with 
a different murderer at the conclusion of the broadcast. 

[0186] Using the foregoing embodiments, methods and 
processes, the interactive multimedia computer maximizes 
personalized attention and interactivity to subscribers in 
their homes in real time. Although the present invention has 
been described in detail with respect to certain embodiments 
and examples, variations and modifications exist which are 
within the scope of the present invention as defined in the 
following claims. 

What is claimed is: 

1. A live interactive digital programming system, com- 
prising: 

a viewer television reception system for receiving live 
interactive programming and branching codes, the live 
interactive programming comprising a plurality of digi- 
tally compressed video signals and audio signals, the 
reception system comprising: 

a viewer interface for receiving a viewer entry; 

a microprocessor, connected to the viewer interface, for 
selecting one each of the plurality of video signals 
and audio signals and directing a seamless switch to 
at least one of the selected video signal and the 
selected audio signal at a predetermined time, the 
selection of the selected video signal and the selected 
audio signal and the predetermined time of each 
selection a function of at least one of the branching 
codes and the received viewer entry; 

a demultiplexer for demultiplexing the plurality of 
video signals and audio signals; 

a decompressor/decoder connected to the demulti- 
plexer for decompressing the demultiplexed plurality 
of video signals and audio signals; 

a means for displaying the selected video signal; and 

a means for playing the selected audio signal. 

2. The live interactive digital programming system of 
claim 1, wherein the plurality of digitally compressed video 
signals correspond, respectively, to a plurality of different 
predetermined camera angles of a live event. 

3. The live interactive digital programming system of 
claim 1, wherein the live interactive programming further 
comprises a graphics signal wherein the microprocessor 
selects the graphics signal at a predetermined time, the 
selection of the graphics signal a function of at least one of 
the branching codes and the received viewer entry, and 



wherein the live interactive digital programming system 
further comprises a means, connected to the microprocessor, 
for presenting the selected graphics signal on the display 
means. 

4. The live interactive digital programming system of 
claim 1, wherein the display means presents at least one 
interrogatory to the viewer, the content of the interrogatory 
involving program options, and the viewer entry comprises 
a response to at least one interrogatory. 

5. A live interactive digital programming system, com- 
prising: 

a viewer television reception system for receiving live 
interactive programming and branching codes, the live 
interactive programming comprising a plurality of digi- 
tally compressed video signals and audio signals, the 
reception system comprising: 

a memory for storing a viewer profile; 

a microprocessor, connected to the memory interface, 
for selecting one each of the plurality of video 
signals and audio signals and directing a seamless 
switch to at least one of the selected video signal and 
the selected audio signal at a predetermined time, the 
selection of the selected video signal and the selected 
audio signal and the predetermined time of each 
selection a function of at least one of the branching 
codes and the stored viewer profile; 

a demultiplexer for demultiplexing the plurality of 
video signals and the audio signals; 

a decompressor/decoder connected to the demulti- 
plexer for decompressing the demultiplexed plurality 
of video signals and audio signals; 

a means for displaying the selected video signal; and 

a means for playing the selected audio signal. 

6. The live interactive digital programming system of 
claim 5, wherein the plurality of digitally compressed video 
signals correspond, respectively, to a plurality of different 
predetermined camera angles of a live event. 

7. The live interactive digital programming system of 
claim 5, 

wherein the live interactive programming further com- 
prises a graphics signal; 

wherein the microprocessor selects the graphics signal at 
a predetermined time, the selection of the graphics 
signal a function of at least one of the branching codes 
and the stored viewer profile; and 

wherein the live interactive digital programming system 
further comprises a means, connected to the micropro- 
cessor, for presenting the selected graphics signal on 
the display means. 

8. Alive interactive digital presentation system, compris- 
ing: 

a means for receiving live interactive programming from 
a digital program stream, wherein the five interactive 
programming contains a plurality of digital video sig- 
nals and audio signals; 
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a viewer interface for receiving a viewer entry; 

a microprocessor, connected to the viewer interface, for 
selecting and seamlessly switching to one each of the 
plurality of video signals and audio signals based on the 
viewer entry; 

a means for displaying the selected video signal; and 

a means for playing the selected audio signal. 

9. The live interactive digital presentation system of claim 
8, wherein the digital program stream is received from a 
satellite transmission system. 

10. The live interactive digital presentation system of 
claim 8, wherein the digital program stream is received from 
a cable distribution system. 

11. The live interactive digital presentation system of 
claim 8, wherein the digital program stream is received from 
a broadcast transmission system. 

12. The live interactive digital presentation system of 
claim 8, wherein the digital program stream is received 
within a private network. 

13. The live interactive digital presentation system of 
claim 8, wherein the combined digital program stream is 
received within an in-stadium network. 

13. The live interactive digital presentation system of 
claim 8, wherein the system is embodied within a computer 
workstation. 

15. The live interactive digital presentation system of 
claim 8, wherein the digital program stream is received over 
the Internet. 

16. The live interactive digital presentation system of 
claim 8, wherein the plurality of digital video signals cor- 
responds to a plurality of different predetermined camera 
angles. 

17. The live interactive digital presentation system of 
claim 8, wherein one of the plurality of digital video signals 
corresponds to a main program video feed. 

18. The live interactive digital presentation system of 
claim 8, wherein each of the plurality of digital video signals 
corresponds, respectively, to each of the plurality of audio 
signals. 

19. The live interactive digital presentation system of 
claim 8, wherein the programming further contains an 
encoded information segment address, wherein the informa- 
tion segment address specifies the location of an information 
segment, the system further comprising: 

a means for decoding, connected to the receiving means, 
the information segment address; 

a means for retrieving, connected to the decoding means, 
the information segment located at the information 
segment address and; 

wherein the display means presents the selected video 
signal at the same time as or in replacement of the 
information segment. 

20. The live interactive digital presentation system of 
claim 19, wherein the information segment address is a 
uniform resource locator, the uniform resource locator speci- 
fying a World Wide Web site address on the Internet. 

21. The live interactive digital presentation system of 
claim 19, wherein the information address segment specifies 
an entry in a database index accessible via a communication 
network. 



22. A method for providing live interactive digital pro- 
gramming comprising: 

obtaining a plurality of video signals from a plurality of 
video cameras, wherein at least one of the plurality of 
video cameras provide a differentiable view of a live 
event; 

receiving the plurality of video signals in a control studio; 

digitally compressing the plurality of video signals; 

digitally multiplexing the plurality of video signals into a 
combined digital program stream; 

transmitting the combined digital program stream; 

receiving the combined digital program stream at a recep- 
tion site; 

digitally demultiplexing the plurality of video signals; 
digitally decompressing the plurality of video signals; 
selecting at least one of the plurality of video signals; and 

displaying the selected video signal. 

23. The method of claim 22, wherein the combined digital 
program stream is transmitted over a satellite transmission 
system. 

24. The method of claim 22, wherein the combined digital 
program stream is transmitted over a cable distribution 
system. 

25. The method of claim 22, wherein the combined digital 
program stream is transmitted over a broadcast transmission 
system. 

26. The system of claim 22, wherein the combined digital 
program stream is transmitted within a private network. 

27. The system of claim 22, wherein the combined digital 
program stream is transmitted within an in-stadium network. 

28. The system of claim 22, wherein the combined digital 
program stream is transmitted over the Internet. 

29. The method of claim 22, further comprising the steps 
of: 

obtaining viewer specific information; 

creating a viewer profile with the obtained viewer specific 
information; and 

wherein the step of selecting is based, at least in part, on 
the viewer profile. 

30. The method of claim 29, further comprising storing 
the viewer profile in a database. 

31. The method of claim 30, wherein the database is 
located at a site remote from the reception site. 

32. The method of claim 30, wherein the database is 
located at the receive site. 

33. The method of claim 29, wherein the step of obtaining 
viewer specific information further comprises the steps of: 

displaying an interrogatory to the viewer, the content of 
the interrogatory involving program options; 

collecting an entry from the viewer in response to the 
interrogatory; and 

wherein the step of selecting is based, at least in part, on 
the collected viewer entry. 
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34. A method for providing live interactive digital pro- 
gramming, comprising: 

receiving live interactive programming, the live interac- 
tive programming comprising a plurality of digitally 
compressed video signals and audio signals, and at 
least one encoded information segment address speci- 
fying the location of a related information segment; 

demultiplexing the plurality of video signals and audio 
signals; 

decompressing the demultiplexed plurality of video sig- 
nals and audio signals; 

obtaining a viewer entry; 

selecting one each of the plurality of video signals and 
audio signals based, at least in part , on the viewer 
entry; 



directing a seamless switch to the selected video signal 
and the selected audio signal; 

decoding the information segment address; 

retrieving the related information segment at the infor- 
mation segment address; and 

presenting the selected video signal, the selected audio 
signal, and the information segment. 

35. The method of claim 34, wherein the information 
segment address comprises a uniform resource locator, the 
uniform resource locator specifying a World Wide Web site 
address on the Internet. 



* * * * * 
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